Abstract: Visual localization is a fundamental task in computer vision, which is widely used in UAV control, surveillance system and remote sensing. In GNSS denied cases, visual localization using existing image references is an alternative navigation method for UAVs. However, due to the scene differences caused by changes of weather, season and illumination, the accuracy of visual localization of UAV images can hardly be guaranteed, especially when the UAV is flying close to the ground. In this paper, a visual localization framework with image synthesis is proposed to solve these problems. The proposed framework combines shadow mappingbased texture fusion and deep convolutional inpainting network to synthesize novel view images. These synthesized images can be used as additional reference data to solve the appearance change and large parallax problem, by improving the accuracy of 2D 3D feature mapping and registration in the stage of pose estimation. Experimental results show that the performance of proposed image synthesis is better than traditional stitching methods, in terms of visualization, number of matches, confidence and localization accuracy. It is proved that the proposed method can support UVA visual localization with large appearance changes.

Key words: visual localization, UAV, image synthesis, 3D reconstruction